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1. INTRODUCTION 
***<•****★★*********★ 

PRC Speas, assisted by David R, Bornemann Associates, Inc., has conducted 
analyses of flight plan data for the National Aeronautics and Space 
Administration-Lewis Research Center under Contract #NAS3-22748. 

The objective of these analyses was t?^ assess the potential improvement 
in fuel savings which may oe possible from improved meteorological 
data. Flight plans calculated from prescribed input parameters and 
meteorological data sets were used as quantitative indicators of differ- 
ences in fuel burn and other relevant parameters. Flight plan data were 
provided through the cooperation of two airlines which will be referred 
to as "BLUE Airlines" and "RED Airlines" throughout this report in order 
to maintain anonymity. 

The work program under this contract was divided into four tasks, each of 
which involved various comparisons of flight plans, flight tracking, or 
wind and temperature data developed from weather forecasts or actual 
analyses under different sets of conditions. The final report consists 
of four volumes, each presenting the findings for one of the tasks, and 
this volume which summarizes the entire study and its findings. 
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2. SUMMARY OF RESULTS 
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2.1 OVERVIEW 

The results of this study have shown that the potential fuel savings from 
using more timely and accurate weather data in flight planning and route 
selection are conservatively estimated to be between 1.2 and 2.5 percent. 
This occurs when comparing a flight plan based on the operational 
Suitland forecast and on a route compatible with the North Atlantic 
tracks with one based on the actual weather (verifying analysis) using 
the minimum fuel track routing. Further results from actual flights show 
that savings may be greater. 

In order to verify the accuracy of the Suitland forecast, flight plans 
developed on it were compared with actual data observed by the aircraft 
(and averaged over 10 degree segments). The results showed that the 
average difference between the forecast and observed wind speed was 9 
kts. without considering direction, and the average difference in the 
component of the forecast wind parallel to the direction of the observed 
wind was 13 kts. - both Indicating that the Suitland forecast under- 
estimates the wind speeds. The Root Mean Square (RMS) vector error was 

30.1 kts. The average absolute difference in direction between the 
forecast and observed wind was 26 degrees and the temperature difference 
was 3*C. These results Indicate that the forecast model as well as the 
verifying analysis used to develop comparison flight plans was a limiting 
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factor and that tne average potential fuel savings or penalty are up to 
3.6 percent depending on the direction of flight. These potential 
savings are entirely attributable to improved forecast accuracy since the 
routes were the same. 

Data are also developed which show that the track selection methodology 
of many airlines operating on the North Atlantic may not be optimum and 
that the interpolation process (or lack of it) in developing proper wind 
and temperature fields is an important source of error. 

Additional data show that the most fuel efficient routes between South 
America and Europe do not include the fixed ATC track. 

An in-depth analysis of differences between the forecast and verifying 
analysis during the 33 day test period was conducted. Fifteen of the 33 
days showed significant errors in the operational forecast. On these 
days forecast wind speeds in the area of maximum winds were in error by 
at least 20 to 25 kts. and sometimes by more than 50 kts. All of these 
days were in the summer or fall and one might expect errors of larger 
magnitude during the winter. 

2.2 OBJECTIVES AND METHODOLOGY 

This study consisted of four data analysis tasks. 
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« Task I compared fuel burn, flight time, air miles and ground miles 
on flight plans that were based on the U.S. National Weather Service 
(NWS) forecasts with corresponding plans based on the NWS verifying 
analyses valid at the time of the forecasts to determine the fuel 
savings that were possible through improved weather forecasts. The 
comparisons involved flight plans that were produced on the RED and 
BLUE Airlines flight planning systems. 

• Task II consisted of similar comparisons to those in Task I but was 
based on flight plans and flight tracking data that were produced by 
a different system developed at SRI International for the Federal 
Aviation Administration, The flight tracking data simulated actual 
flight operations and accounted for flight plan deviations such as 
those required by air traffic constraints. 

• Task III compared segment winds and temperatures from flight plans 
based on NWS forecasts to the actual winds and temperatures observed 
by aircraft that operated those flights. 

• Task IV used flight plan segment wind and temperature differences 
from Task I as indicators of forecast errors which were then 
analyzed further to determine their probable cause, extent and 
significance. 


Even though the data sources and methodology differed in each Task, 
all four tasks resulted in conclusions that clearly suggested there 
is potential for further fuel savings through more accurate weather 
forecasts and flight plans. 


2.3 KEY FINDINGS 

Some key findings from the various tasks are summarized below: 


• Task I comparisons between B747 flight plans on operational routes 
(which were probably not minimum fuel tracks) based on the NWS 
forecast and plans on the same routes based on the verifying 
analysis showed that fuel savings per flight on eastbound North 
Atlantic routes would average 462 kg (151 gal.) if the forecasts 
and verifying analyses were identical, or in other words, if 
the forecasts were perfectly accurate. Savings for westbound 
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flights were negligible. Differences in fuel burn between the plans 
on the forecast and the plans on the verifying analysis were always 
positive eastbound and negative westbound, suggesting that wind 
speeds are always underestimated in the forecast. 

« Further Task I analyses showed that if operational constraints on 
the use of minimum time tracks and optimum flight levels were not 
imposed by the ATC system the potential savings per flight increases 
to 1048 kg (345 gal.) or 1879 kg {618 gal.) depending on the 
direction of flight. 

t Based on analyses of tracks selected in 144 cases it was determined 
that airlines whose track selection methodology is based on minimum 
time rather than minimum fuel, and airlines that select a prelimi- 
nary minimum time track at a constant flight level before optimizing 
the flight level, do not select the optimum fuel NAT track 45 to SO 
percent of the time, 

• Comparisons of track selections between Amsterdam and Caracas 
on a single fixed route that had been used by one airline and on 
eight other routes laid out by NASA showed that the original route 
was optimum only six times out of 60 cases. The average fuel burn 
penalty for being on this route was 1,054 kg (347 gal.) per flight. 

• Task II comparisons of flight plans based on the NWS forecast with 
corresponding plans based on the verifying analysis but using the 
SRI model, resulted in the comparable finding that fuel savings of 
364 kg (122 gal.) were possible for eastbound B747s on existing 
operational North Atlantic routes if the forecasts were equal to the 
verifying analysis. 

• Task II comparisons involving flight tracking data found that actual 
fuel burn and flight times were always higher than planned, in 
either direction, and even when the same weather data set was used. 
This suggests that either there is an error in the flight tracking 
algorithm or that a penalty is incurred due to traffic congestion 
and resolution of ATC system conflicts in addition to the penalty 
incurred from inaccurate weather forecasts. Since the flight 
tracking model output resulted in more diversions than is known to 
be the case, it was concluded that there is an error in the flight 
tracking algorithm. 

« Based on analysis of 2,430 flight segments when the flight plan 
winds and temperatures based upon the NWS forecast were compared in 
Task III to actual data observed by the aircraft, the average 
difference between the forecast and observed wind speed was 9 kts., 
the direction difference averaged 26 degrees and the average 
temperature difference was S'C. The average difference in the 
component of the forecast wind parallel to the direction of the 
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observed wind v»as 13 kts. In the worst case {when the wind is 
always from the direction with maximum impact on the direction of 
flight) this 13 kt. error results in fuel burn and time penalties of 
up to 15 minutes and 2,835 kg (932 gal.) of fuel for the average 
B747 flight. These flight segments were from operational routes 
which were not minimum fuel tracks and may not have been in the area 
of maximum wind and thus actual errors may be larger. 

t Although BLUE and RED Airlines data agreed that the absolute value 
of the average temperature errors was 3‘C, BLUE found the temper- 
atures warmer than forecast while RED found them colder than 
forecast. Similarly, BLUE found the wind direction forecast error 
toward decreasing azimuthal directions while RED found errors toward 
increasing azimuth eastbound and decreasing westbound, indicating 
that weather data interpolation errors probably exist in one or both 
flight planning systems. 

• The Task IV analysis showed that significant forecast errors existed 
on 15 of the 33 days included in the study. In particular; 

- Wind speeds were underestimated by at least 20 to 25 kts. and 
sometimes up to 50 kts. in the area of maximum wind on 14 days. 
Speeds were underestimated on most other days, as well, iut to 
a lesser extent. 

There is a tendency to repeat the same forecast errors from 
prog to prog even though the intervening analysis showed 
clearly that the errors had occurred. 

The Task IV analysis also showed that some perceived forecast 
errors may be the result of inadequacies in the weather data 
interpolation techniques used in the airline computer flight 
planning systems. 


In Tasks I and III findings were presented separately for data based on 
the BLUE and RED flight planning systems which had different sample 
sizes. The figures given above represent weighted averages of the 
results from both data sources. 
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3. BACKGROUND 
****************** 

The work conducted in this study was in support of the NASA Commercial 
Aircraft Fuel Savings Program which was developed to investigate the 
potential fuel savings which could be provided by an improved meteoro- 
logical data base. Other portions of the program have investigated the 
impact of various factors, such as data collection procedures, on the 
quality and timing of the meteorological forecasts. 

The Commercial Aircraft Fuel Savings Program was conceived by MSA in 
1978 partly to take advantage of numerous additional data that were then 
being collected as a result of a major international global experiment 
that was being conducted at that time. Prior to that ti..ie, sufficient 
data were never available on a global scale to study systematically the 
potential improvement that could result were significant quantities of 
additional data added to the analysis and forecast process. 

During this global experiment in 1979 a number of DCIO and B747 aircraft 
were equipped with Aircraft Integrated Data Systems (AIDS). AIDS is an 
onboard data processing and storage system which is used by airlines to 
monitor aircraft and engine performance data. For this experiment AIDS 
was used to collect data on position, altitude, temperature, wind 
velocity and time. The data were stored on magnetic tape for ground 
processing. 
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Since U.S. and foreign carriers cooperated 1n the experiment, AIDS 
flights operated throughout the world, although the largest concentration 
of additional data were collected in the North Atlantic area. 

On certain days during 1979 NASA collected AIDS data and corresponding 
airline flight plans for the subsequent analyses conducted in this 
project, AIDS data were provided to the U.S. National Weather Service 
which reran Its nuriie^lcal analysis model for those days to provide a new 
analysis output which had been enhanced by the additional AIDS data. 
NASA also retained on magnetic tape (and in hard copy) copies of the 
original operational forecasts issued on those days to airlines for 
flight planning. 

The plan was to use the airline flight plans as a sensitive indicator of 
the differences between various meteorological data sets, measuring fuel 
burn and flight time or winds and temperatures. Comparisons could be 
made, for example, between the original operational forecast and the AIDS 
enhanced analysis valid at the time of the forecast or between flight 
plans based on the forecast and the raw AIDS data. It was expected that 
the study would show that enhancing the analysis with the AIDS data would 
result in improved forecasts and, in turn, more accurate flight plans and 
fuel savings. 

Much of the data used in this study were developed later by NASA through 
the cooperation of the RED and BLi.c Airlines. NASA provided these 
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airlines with magnetic tape copies of the original NWS operational 
forecasts and analyses reruns, In effect, simulating a real time trans- 
mission of the current Aviation Digital Data. The airlines were also 
supplied with flight plan Inputs, prepared by NASA, and designed to 
control variables such as routings and aircraft weights so as to elimi- 
nate unwanted effects of these variables that are not solely the result 
of the weather data. 

The flight plans calculated on these data by the RED and BLUE Airlines, 
using raw AIDS data for positional and altitude Information, the original 
operational flight plans used on those days In 1979 for the AIDS flights, 
and flight plans produced by the SRI International flight planning and 
flight tracking simulation model were all retained on magnetic tape and 
provided as Input to the analyses conducted In this project. 
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4. DISCUSSION OF TASK I RESULTS 


Task I involved various comparisons of groups of flight plans that were 
based on the operational NWS forecast (the Nine Level Primitive Equation 
Model) and plans that were based on the NWS verifying analysis (the 
Flattery Analysis Model) valid at the time of the forecast. All of 
the flight plans used in the comparisons were produced by the RED or BLUE 
Airlines flight planning systems based upon inputs prepared by NASA. 
These inputs exercised optional control over such parameters as routing, 
flight level, cruise speed and payload so that the affect of these 
variables could be isolated or eliminated and differences between the 
flight plan groups could be attributed properly to differences in the 
weather data. Ten to 20 of the routes were selected to correspond to 
actual flights that were operated on that day by aircraft equipped with 
an Aircraft Integrated Data System (AIDS). (AIDS provided for collection 
and storage of actual weather observation data on magnetic tape for 
subsequent processing.) The remaining flight plans were computer 
generated and did not necessarily represent any flights actually flown. 

Weather and flight plan data sets from 31 days in August through November 
and two from January, were used in the Task I analysis. In order to 
randomize the data set selection, data from all Mondays and Fridays 
during this period were used. 
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Four categories of flight plan types were considered in Task I. These 
were : 

(1) Flight plans based on operational forecasts on routes that were 
flown by AIDS equipped aircraft; 

(2) Flight plans based on operational forecasts on minimum time/minimum 
fuel routes; 

(3) Flight plans based on tfe verifying analysis valid at the times of 

the forecasts in (1) and {2} and on the same routes used in (1); 

(A) Flight plans based on the verifying analysis valid at the times of 
the forecasts in (!) and (2) but on the minimum time/minimum fuel 

track determined from the verifying analysis. 

Differences in fuel burn, flight time, air miles, ground miles and the 
ratio of air miles to ground miles for B747 aircraft were determined for 
the following five comparisons between the above groups: 

Case 1 - Group 1 minus Group 3 

Case 2 - Group 2 minus Group 4 

Case 3 - Group 1 minus Group 2 

Case 4 - Group 3 minus Group 4 

Case 5 - Group 1 minus Group 4 

In addition to the above comparisons, an analysis was conducted of track 
selections by the RED and SLUE Airlines flight planning systems from the 
available North Atlantic Organized Tracks between New York and Amsterdam^ 
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An analysis of track selections from a group of fixed tracks between 
Amsterdam and Caracas was also conducted. 

The principal findings for each case are summarized In the following 
subsections. The detailed findings are presented in Volume I of the 

final report by direction of flight, region of the world, and data source 
(RED or BLUE flight planning system), and Include the standard deviation, 
variance, 90 percent confidence limits and mean value for each group of 
comparisons. Since the sample sizes vary widely the findings for some 
regions are less significant statistically and may be misleading. 
Therefore, only the fuel burn results for the North Atlantic and for the 
total will be presented here. In some of the comparisons all, or nearly 
all, of the plans are for North Atlantic flights. 

4.1 CASE 1 

The first case compared recreated AIDS flights on the forecast weather 
to recreated AIDS flights on the verifying analysis. Since all other 
flight plan parameters . (route, flight level, etc.) were held constant, 
differences between the flight plans reflected differences between the 
weather data sets and. In this case, the potential fuel savings that 
would result if the forecast and analysis were equal, or in other words, 
if the forecast were perfectly accurate. 
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The number of flight plan comparisons in the sample and the average 
differences (forecast plan value minus actual plan value) in fuel burn 
were: 


SLUE System Flight Plans 


RED System Flight Plans 

All eastbound flights 
All westbound flights 


Sample Burn 

Size Difference 


222 369 kg 
109 569 kg 
306 -140 kg 
143 -295 kg 


Sample Burn 

S i ze Difference 


95 384 kg 

147 -371 kg 


All eastbound flights 

North Atlantic eastbound flights 

All westbound flights 

North Atlantic westbound flights 


The positive differences eastbound, and negative differences westbound 
indicated that wind spe<^ds are generally underestimated in the NWS 
forecast model . 


4.2 CASE 2 

Case 2 compared minimum time tracks on the forecast to minimum time 
tracks on the verifying analysis and, as such, was quite similar to Case 
1 except for the use of different routes. The objective was to determine 
the potential fuel savings that would result if the forecast were perfect 
and if carriers could use random tracks. 
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The number of flight plan comparisons in the sample and the average 
differences in fuel burn in this case were: 


BLUE System Flight Plans 

Sample 

Size 

Burn 

Difference 

A1 1 eastbound 

229 

739 kg 

North Atlantic eastbound 

199 

815 kg 

All westbound 

231 

-409 kg 

North Atlantic westbound 

202 

-322 kg 


Sample 

Burn 

RED System Flight Plans 

Size 

Difference 

All eastbound 

29 

475 kg 

All westbound 

29 

-324 kg 

CASE 3 




Case 3 compared flight plans on routes actually flown by AIDS flights on 
the forecast weather to the corresponding minimum time track on the 
forecast weather. The objective was to show the potential fuel savings 
that could result if carriers were free to fly the minimum time track. 


Based on the raw data the sample sizes and the average differences or 
savings in fuel burn were; 


SLUE System Flight Plans 


Sample Burn 

S i ze Difference 


Eastbound North Atlantic 
Westbound North Atlantic 


36 511 kg 

49 1978 kg 
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Sample 

Burn 

RED System Flight Plans 

Size 

Difference 

Eastbound North Atlantic 

7 

18)2 kg 

Westbound North Atlantic 

10 

I'/oO kg 


Since this case compared recreated AIDS flights to minimum time tracks, 
the BLUE data were distorted due to the use of direct routings from 
origin to oceanic entry point and due to flight level differences (See 
Section 8.2 for further discussion). Estimates of the effect of these 
routing and flight level differences were computed and the estimated BLUE 
differences after these adjustments were applied were: 


Burn 

Difference 

Eastbound 1061 kg 

Westbound 1397 kg 

4.4 CASE 4 

Case 4 was identical to Case 3 except that both the AIDS flights and 
minimum time tracks were based on the verifying analysis. 

Fuel burn differences using the raw data were: 


Sample Burn 

BLUE System Flight Plans Size Difference 

Eastbound North Atlantic 35 

Westbound North Atlantic 48 


666 kg 
2096 kg 
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Sample 

Burn 

RED System Flight Plans 

Size 

Difference 

Eastbound North Atlantic 

7 

1931 kg 

Westbound North Atlantic 

10 

1937 kg 


After applying an adjustment for the routing and flight level differ- 
ences, the BLUE savings were: 


Burn 

Difference 

Eastbound 1216 kg 

Westbound 1515 kg 


4.5 CASE 5 

Case 5 combined the conditions of Cases 3 and 4 and compared AIDS flights 
on the forecast weather to minimum time track flights on the verifying 
analysis. The objective was to show the combined savings from improved 
forecasts and from eliminating ATC restrictions on the use of the 
minimum time track. 


Again, based on the raw data alone, the potential savings were: 


BLUE System Flight Plans 

Eastbound North Atlantic 
Westbound North Atlantic 


Sample Burn 

Size Difference 


36 1311 kg 

49 1594 kg 
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Sample 

Burn 

RED System Flight Plans 

Size 

Difference 

Eastbound North Atlantic 

6 

1989 kg 

Westbound North Atlantic 

10 

1220 kg 


Applying the adjustment to the BLUE data results in savings of: 


Burn 

Difference 

Eastbound 1861 kg 

Westbound 1013 kg 


4.6 NORTH ATLANTIC TRACK SELECTION 

Analysis of flight plans run on each of the North Atlantic Organized 
Tracks at each available flight level on each of 30 days in both direc- 
tions tested three hypotheses, each of which is followed by some airlines 
and is incorporated in their track selection algorithms. 


1) On 40 out of 84 “days" (30 eastbound and 30 westbound for BLUE, and 
16 eastbound and 8 westbound for RED) the minimum time track was not 
coincident with the minimum fuel track. Thus, airlines whose track 
selection is based on time rather than fuel are on the wrong track 
45 percent of the time. 

2) In 28 out of 60 cases the minimum time track selected at FL330 or 
FL350 was not the minimum fuel track at optimum altitude. This 
indicates that airlines that select a preliminary minimum time track 
at a constant flight level and then optimize for fuel in the flight 
plan are on the wrong track 47 percent of the time and incur an 
average fuel burn penalty of 248 kg each time they are on the wrong 
track. (This is based on comparisons of flight plans using the NWS 
models ope'^ational in 1979. Data from other tasks in this study 
indicate that the penalty would be larger if actual data were 
compared.) 
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3) In 19 out of 60 cases the minimum fuel track on the verifying 
ar '.lysis was not the same track as the minimum fuel track on the 
forecast. This is contrary to the beliefs of many carriers who feel 
that the weather changes so slowly that the best track does not 
change between the forecast and the verifying analysis even though 
the time and fuel burn on that track might change. 


4.7 AMSTERDAM-CARACAS TRACKS 

One airline, and possibly more, had been using a single fixed route 
between Europe and the Caribbean. NASA laid out eight additional routes 
and ran flight plans on each one, in each direction for 30 days. 


The original fixed route proved to be the best one only six times out of 
the 60 cases. The average fuel burn penalty for being on this route was 
1054 kg per flight. 


% 
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5. DISCUSSION OF TASK II RESULTS 

In Taste II comparisons were conducted between various categories of 
flight plans and flight tracking data that were produced by a simulation 
system developed ’at SRI International for the Federal Aviation Adminis- 
tration. Based upon a given set of weather data the system produced 
flight plans and flight tracking data. Flight tracking data simulate the 
actual flight tracks of all aircraft operating on a given weather data 
set and provide such features as the rerouting of some flights as 
necessary to resolve ATC conflicts. The weather data used were the same 
NWS forecasts and verifying analyses used in Task I but covered fewer 
days. 

Four categories of data were analyzed. These were: 


(1) Flight plans based on an operational forecast; 

(2) Flight tracking based on the flight plans in (1) but using the 

weather from the verifying analysis valid at the time of the 
forecast; 

(3) Flight plans based on the actual weather valid at the time of the 
forecast used in (1); 

(4) Flight tracking based on the flight plans in (3) and using the 

actual weather from the verifying analysis. 
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As in Task I comparisons were made of differences in fuel burn, flight 
time, air miles, ground miles and the ratio of air miles to ground miles 
between the following flight plan and flight tracking categories; 

Case 1 - Group 1 and Group 2 
Case 2 - Group 1 and Group 3 
Case 3 - Group 2 and Group 4 
Case 4 - Group 3 and Group 4 

Case 5 - Groups 1 and 3 with the actual airline flight plans from 
Task I 

Comparisons were made for entire flights and for flight segments and were 
presented by direction of flight, region and by aircraft type groups such 
as 8747s, DClO/LlOlls, or B707/DC8S. The detailed findings are presented 
in Volume II* of the final report. 

The results of the comparisons for 8747 aircraft for each of the five 
cases are summarized in the following subsections. Findings for other 
aircraft type groups were comparable but only the 8747 data are presented 
here for consistency in comparisons to the Task I findings which were 
exclusively for 8747s. 

5.1 CASE 1 

The first set of comparison data were developed by subtracting the 
flight tracking values on the verifying analysis from the flight plan 
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values developed on the forecast weather. The conditions in this case 
were similar to those in Case 1 of Task I in that the comparison measured 
potential fuel savings that could result if the weather forecast were 
equal to the verifying analysis. Only the model, or source of the 
flight plans, was different and the actual effect of ATC diversions was 
included. 

Tne numbers of B747 flight plan comparisons in the sample and the average 
differences in fuel burn were: 


Sample Burn 

Size Difference 


Eastbound North Atlantic 

167 

-775 kg 

Eastbound Polar 

36 

-810 kg 

Westbound North Atlantic 

235 

-1278 kg 

Westbound Polar 

39 

-451 kg 


The negative values eastbound are contrary to the Task I findings and 
would imply that wind speeds are always overforecast, or that the flight 
tracking fuel burn is always higher because of ATC diversions. 

For westbound flights these findings are consistent with the Task I 
results but they are contrary to the eastbound results and imply that 
aircraft always burn more than flight plan regardless of whether they are 
flying against or with a wind forecast error. One must conclude that 


SPEAS 


22 . 


the differences are not entirely weather related and must be greatly 
influenced by the conflict resolution algorithm of the flight tracking 
model . 

5.2 CASE 2 FINDINGS 

In the second case flight plans on the forecast weather were compared 
to flight plans on the verifying analysis. Except for the use of a 
different flight planning model as the data source and except for the 
fact that new NAT tracks were selected on the verifying analysis, and 
thus routings could be different, this analysis was also similar to Case 
1 of Task I. 

The numoers of B747 flights compared and the average fuel burn differ- 
ences for Case 2 were: 



Sample 

Burn 


Size 

Difference 

Eastbound North Atlantic 

159 

374 kg 

Eastbound Polar 

33 

318 kg 

Westbound North Atlantic 

154 

-237 kg 

Westbound Polar 

24 

-420 kg 


The positive differences eastbound and negative differences westbound 
were consistent with and confirmed the Task conclusion that wind 
speeds were normally underestimated, and indicate that negative data 
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sets In Task II Case 1 were probably the result of the flight tracking 
algorithm. 

5.3 CASE 3 FINDINGS 

Case 3 comparisons were developed by subtracting the times, burns and 
other parameters on the flight tracking data from Group 4, based on the 
verifying analysis, from the corresponding Group 2 flight tracking data 
which were also based on the verifying analysis but used flight plan 
Inputs based on the forecast. 

Since the same weather data were used in each case this comparison was a 
measure of the potential fuel savings (or penalty) that could result from 
improvements in the flight plan or track inputs to the flight tracking 
simulator. 

* 

The findings for B747 comparisons in Case 3 were: 


Sample Burn 

Size Difference 


Eastbound North Atlantic 

159 

273 kg 

Eastbound Polar 

33 

96 kg 

Westbound North Atlantic 

154 

243 kg 

Westbound Polar 

24 

-79 kg 
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Positive values, both eastbound and westbound, suggest that either fuel 
burn penalty from the ATC system is less when an improved forecast is 
used in flight planning or that errors were introduced by the flight 
tracking algorithm. 

5.4 CASE 4 FINDINGS 

Case 4 compared data from flight plans based on the verifying analysis 
to flight tracking data developed from the same verifying analysis. 

Since the same weather was used, differences found between these two 
groups of plans were unrelated to weather but represented a measurement 
of the potential effect of improved NAT track selection and the conflict 
resolution simulations of the flight tracking model. 

The findings for the B747 comparisons in Case 4 were; 


Sample Burn 

Size Difference 


Eastbound North Atlantic 

160 

-927 kg 

Eastbound Polar 

33 

-1034 kg 

Westbound North Atlantic 

154 

-599 kg 

Westbound Polar 

24 

-76 kg 


The negative differences in both directions suggest that actual flight 
times, fuel burns and a.ir miles are always greater than planned even when 
the plans were based on the verifyirsg analysis, and that these penalties 
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are the result of conflict resolution and traffic congestion in the ATC 
system or in the flight tracking algorithm. 

5.5 CASE 5 FINDINGS 

The objective of the final comparison in Task II was to determine the 
flight parameter differences between the flight plans developed by the 
SRI model for this task and the corresponding actual airline flight 
plans from Task III. Hoi-<ever, since takeoff weights, flight levels and 
routings were quite different, average burn differences of more than 
10,000 kg per flight were found. Since these variables could not be 
controlled, it would be difficult to attribute the differences to any 
particular cause and these data were judged to be of relatively little 
value. 
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6. DISCUSSION OF TASK III RESULTS 

The Task III analysis compared actual wind ind temperature observations 
taken by AIDS equipped aircraft during eight months of 1979 with data 
from the Flight plans used by those flights. The flight plans were based 
upon the NWS forecasts valid near the time the flights operated and were 
computed on the RED or BLUE Airlines flight planning systems. Flight 
plans for flights of the other airlines that use the BLUE Airlines system 
were also included in this analysis. 

The objective of the Task III analysis was to determine differences that 
existed between the forecast winds and temperatures and those actually 
observed by the aircraft. In Task I, differences were determined 
(measured by fuel burn, flight time and air miles) between the forecast 
and the actual as represented by the NWS forecast and analysis models, 
after the data were subjected to the editing, smoothing, and other 
adjustments inherent in the model. 

Computer programs were developed to extract the wind and temperature data 
from the flight plans and AIDS tapes, reduce them to comparable flight 
segments, and produce statistics on the differences between the forecast 
and actual winds and temperatures. While the flight plan winds and 
temperatures were normally already available as averages of ten degree 
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I 

i 

i 

i 
1 

longitude segments, the AIDS observations were typically spaced at 200 km |j 

■1 

intervals and averages for 10 degree segments had to be developed. i 

A cubic spline function* was used to represent the AIDS flight's wind j 

direction, speed, temperature and latitude as a continuous function of I 

longitude along the flight path. The discrete values of these parameters 
were then determined for each five degree meridional crossing - i.e., 

SOW, 45W, 40W, 35W, etc. Average values for the ten degree segment 

were then determined weighting the midpoint twice the weight of each 

endpoint. 

Segment data were checked for matches of month, day, flight number, 

origin, destination and flight level. Segments for which the AIDS ' 

flight did not match the operational flight plan were discarded. 


6.1 MAGNITUDE OF FORECAST DIFFERENCES 


Matching data from AIDS flights 

and flight plans 

were found 

segments distributed regionally as 

follows: 



BLUE 

RED 


Airlines 

Airlines 

Eastbound North Atlantic 

394 

358 

Westbound North Atlantic 

696 

736 

Eastbound Polar 

4 

79 

Westbound Polar 

13 

109 

Eastbound Mid -Atlantic 

9 


Westbound Mid-Atlantic 

32 
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Differences between the flight plans and AIDS data per flight segment, 
and thus between the forecast and the observed, are summarized below: 

BLUE RED 

Airl ines Airlines 

Average Algebraic Difference 


Wind Direction 
Wind Speed 
Temperature 

Average of Absolute Values of Differences 

-5 deg 
-9 kts 
V2"C 

+1 deg 
-5 kts 
+1’C 

Wind Direction 

29 deg 

20 deg 

Wind Speed 

14 kts 

13 kts 

Temperature 

3*C 

3“C 

Average Difference in Component of 
Forecast Wind Parallel to Observed Wind 

16 kts 

8 kts 

RMS Vector Error 

33 kts 

24 kts 


Using data from Task I on average North Atlantic flight times and fuel 
burns, it was determined that in the worst case (when the wind is always 
from the direction with maximum impact on the direction of flight) this 
error results in fuel burn and time penalties of up to 15 minutes and 
2,835 kg of fuel for a B747 flight. This fuel burn penalty, or potential 
savings if the forecast were perfect, amounts to 3.6 percent of the fuel 
burn for the flight. 

Since the criteria for matching AIDS flights and flight plans resulted 
in some 70 to 80 percent of segments being rejected, a supplemental 
analysis was conducted with relaxed criteria to expand the size of the 
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sample. In this second run segments with flight level differences of 
plus or minus 2,000 feet between the flight plan and AIDS data were not 
rejected. This resulted in a sample of 1,788 BLUE Airlines segments and 
1,282 RED Airlines segments for a total of 3,070 segments. Even though 
the sample increased by 72 percent the average forecast error only 
changed by 1,08 degrees on wind direction, 0.1 kts on speed, and 0.34*C 
on temperature, leading to the conclusion that the original sample was 
large enough to be representative of the real world even though many data 
had to be rejected. 

6.2 SIGN OF FORECAST DIFFERENCES 

Since the average algebraic differences between the forecast values and 
the observed values were determined by subtracting the observed value 
from the forecast value, the algebraic sign of the differences provided 
further data on the forecast errors. 

For the North Atlantic region the means of the algebraic differences 
between the operational flight plan and the AIDS data were: 


Temperature Wind Speed 

( *C) Direction (deg) (kts) 


BLUE Airline 


Eastbound 

-2.28 

-5.35 

-8.27 

Westbound 

-2.53 

-4.17 

-8.97 

RED Air! ine 

Eastbound 

+1.97 

+1.94 

-6.39 

Westbound 

+1.69 

-2.04 

-4.24 
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For both airlines the average wind speed differences were always nega- 
tive, meaning the AIDS winds were stronger and implying that wind speeds 
were underestimated which confirms the findings of the other tasks in 
this study. 

Negative temperature differences mean temperatures were warmer than 
forecast and the findings on temperature were somewhat incongruous. 
Even though both airlines' data agreed that the temperature forecasts 
were in error by approximately 3*C, the BLUE Airlines data showed the 
temperatures warmer than forecast while the RED data implied temperatures 
were colder than forecast {positive differences). Regarding wind 
direction, negative differences mean the forecast wind is from a lower 
azimuthal direction than the ac^'jal wind, or in other words, considering 
that the average wind direction should be from 270 degrees, negative 
differences suggest actual winds more northwesterly than forecast 
and positive differences suggest actual winds more southwesterly than 
forecast. On wind direction, the BLUE Airlines differences were always 
negative while the RED Airlines differences (for the larger North 
Atlantic sample) were positive eastbound and negative westbound. Mo 
information available to PRC Speas suggests an explanation for these 
latter two incongruous findings and it is suspected that they are the 
result of features peculiar to the weather data interpolation techniques 
in use by the RED or BLUE Airlines or both. 
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1 , DISCUSSION OF TASK IV RESULTS 


7,1 OBJECTIVE AND ANALYSIS METHODOLOGY 

In the Task IV analysis, differences between flight plan winds and 
temperatures based on the forecast and on the verifying analysis from 
Task I were used as Indicators of dates and geographic areas for which 
significant forecast errors may have occurred. The objective of Task IV 
was to review available forecasts and verifying analyses valid at the 
forecast time so as to explain forecast errors which occurred and to 
discuss their significance. 

Flight plan segments with wind direction differences greater than 
30 degrees, speed differences greater than 20 kts., or temperature 
differences greater than 5*C, between the plan on the forecast and the 
plan on the analysis, were designated as segments with forecast errors. 
More stringent criteria were tried at first but it resulted in some 
19,000 forecast error segments which were considered to be too numerous 
to be analyzed with meaningful results. 

Many of these "error segments" were widely scattered or isolated geo- 
graphically and of little significance. Others were clustered and 
consistent with other adjacent error segments. Since the objective of 
Task IV was to comment on significant forecast errors, the isolated error 
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segments were Ignored, while those that appeared In patterns of synoptic 
scale or large areal extent were identified for further analysis. 

7.2 IDENTIFICATION OF FORECAST ERRORS 

For those dates and geographic regions so identified, other data pro- 
vided by NASA were reviewed. These normally consisted of the National 
Meteorological Canter (NMC) 250 mb. forecasts and analyses valid at OOOOZ 
on the day under review and OOOOZ on the following day. 250 mb. level 
analyses prepared by Delta Airlines were also used for some days. 

The RED and BLUE Airlines data from Task I initially identified forecast 
errors on 31 of the 33 days included in the study. In Volume IV of the 
final report a summary is presented for each day which gives the number 
of RED and BLUE Airlines flights, the number of error segments, the areas 
in which forecast errors occurred and a discussion of the extent and 
significance of the errors. 

Further review of the data revealed an inconsistency in the RED and BLUE 
Airlines data from Task I. 

A substantially higher number of error segments were found by RED 
Airlines even though it operated fewer flights. For the BLUE Airline 
there were 853 flights and 908 error segments for an average of 1.06 per 
flight. For the RED Airline 262 flights encountered 858 error segments 
for an average of 3.2 per flight. If the Mid East segments are excluded 
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the BLUE Airlines error segments are reduced to 528 for 651 flights, or 
an average of 0.8 per flight. 

Since both airlines used the same weather data, it is obvious that there 
are differences in their interpolation algorithms since only one detected 
apparent "forecast errors" when both operated f 1 ights through the same 
area. 

Further investigation showed that the RED Airlines winds were often 
completely incompatible with either the forecast or the verifying 
analysis. Wind direction errors of up to 180 degrees occurred on days, 
and in geographic areas, on which the SLUE Airlines data showed few or 
zero errors. 

It was decided that some error in the data interpolation or flight 
planning algorithm in the RED system was causing an apparently erroneous 
identification of error segments. 

When the BLUE data alone were considered, 10 of the 33 days were found to 
have forecast errors of significant magnitude and areal extent. For 
three additional days, the BLUE Airlines error segments indicated 
relatively minor forecast errors but, upon visual inspection of the 
forecast and analysis charts, it was determined that significant errors 
existed but were not detected by the BLUE Airline flights which were not 
routed near the forecast error area. 
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Even though most of the forecast errors indicated by the RED flight plans 
were discarded because they were not consistent with the other data, on 
two days the errors indicated by RED Airlines were confirmed by visu-l 
inspection of the charts and were judged to be significant. 

Thus, between the RED and BLUE data, 15 of the 33 days were found to have 
significant forecast errors. 

7.3 FINDINGS 
7,3.1 Forecast Errors 

Data for the 15 days identified two types of repetitive and significant 
errors. These were underestimated wind speeds, and repetition of 
forecast errors. 

Underestimated Hind Speeds 

A persistent error found throughout the analysis was the underestimation 
of wind speeds. In 14 of the forecast error situation the forecast 
maximum winds were at least 20 to 25 kts. and sometimes more than 50 kts. 
lighter than actual. Wind speeds further away from the core were 
proportionately in error out to the 70 kt. or 50 kt. level where the 
errors become insignificant. Since the actual value of the maximum wind 
speed and the lateral extent of the "significant" or stronger winds 
varied from day to day it is not possible in summary to quantify this 
finding further. 
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Wind speed forecast errors were also found on five additional days but 
these were judged to be less significant errors because of their lesser 
magnitude and extent. 

Repetition of Forecast Errors 

In the analysis it was noted that apparent forecast errors were not 
corrected on subsequent forecasts. Review of the forecasts issued IZ or 
24 hours later showed that errors such as the wind speed errors noted 
above were carried from prog to prog even though the intervening analysis 
showed many actual observation*j of data that were In disagreement /^ith 
the forecast. A forecast, for example, that showed a maximum wind 
isotach of 90 kts. would be followed by another with a 90 kt. maximum 
isotach value even though winds of 125 to 135 kts. were observed at 
the valid time of the prog. Sometimes, the same situation would .have 
occurred 24 hours earlier and 24 hours later, as well, with the forecasts 
giving no indication of the existence of the stronger winds. 

This repetition of the forecast errors was confirmed on five days of the 
33 reviewed. There were indications that forecast errors were repeated 
on many of the other days but it was not possible to confirm this since 
data were only retained for the days involved in the study, and thus, 
data for the adjacent forecast periods were not always available. 
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Location of Synoptic Features 

Even though there were frequent errors In wind speed forecasts, as noted 
above, the forecasts of the location, movement, development and intensity 
of synoptic scale features were usually quite accurate, ‘On five days 
the forecasts were judged to have significant errors of this nature. 
Typically, these consisted of such errors as underestimating the extent 
of deepening of a trough which resulted In an extensive area of wind 
direction errors. On’ three additional days, less significant errors of 
this type were noted. 

7.3.2 Other Errors 

Analysis of the error segments and visual Inspection of the corresponding 
forecast and analysis charts found several other errors which may have 
been perceived as forecast errors but are likely the result of some 
procedure or feature of the airline flight planning systems. These are 
discussed below. 

Time Interpolation and Choice of Prog 

It was apparent from the analysis that time interpolation between 
weather prcgs, or the choice of prog on which the flight plan Is based, 
caused some of the wind differences between the forecast and analysis, 
especially In rapidly changing situations. On several days, review of 
the OOOOZ forecast and analysis, and review of the forecast and analysis 
valid 24 hours later showed that the forecasts were highly accurate, 
but flight plans based on an Intervening forecast, such as the 1800Z, 
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resulted in error segments. It is possible that in these few cases there 
were no forecast errors at all but rather that the flight planning 
algorithm or the analysis methodology somehow resulted in forecast winds 
being compared to actual winds valid at a different time. 

Average Hinds in Areas of Sharp Gradient 

The methodology by which these airlines develop the average winds for a 
flight plan segment may have resulted in some "forecast errors". Both 
airlines use some scher. to determine an average wind for a flight plan 
segment from adjacent Marsden Square data. Since some flight plan 
segments can traverse more than one Marsden Square, in areas of sharp 
gradient flight plan winds may be developed by averaging very light winds 
with very strong winds, depending on the algorithm being used. This 
averaging process appears to have resulted in some apparent forecast 
errors on the flight plans, which do not appear when one visually 
compares the forecast and analysis charts. 

This problem applies equally to situations with a pronounced wind 
direction gradient. Many forecast error segments were found near the 
centers of highs, lows, ridges or troughs where the wind direction 
changes rapidly with distance. 

Mid-East Errors 

Significant "forecast errors" were noted on U days in the Mid East area 
by BLUE Airlines. Even on other days there were usually some error 
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segments but they were judged to be relatively insignificant and not 
worth further analysts. Often these "errors'* were on the same routes day 
after day - routes through Yugoslavia, Turkey, Syria, Iraq, and Iran. 
The charts that were available did not extend far enough east to be of 
help in explaining the forecast errors in this area. However, it is 

believed that the persisten', almost daily, occurrence of forecast errors 
in this area is significant on its own even if the significance of the 
individual daily errors cannot be determined. Since it is difficult to 
explain how a forecast error should occur in the same location so often, 
it was assumed that this was the result of some anomaly in the BLUE 
Airlines system. 
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8. METHODOLOGY AMO DATA REDUCTION 
********************** **************** 


8.1 METHODOLOGY 

Numeraus flight plan data were provided by NASA on magnetic tape for the 
analyses conducted during this study. In Task I alone, for example, up 
to 20,000 flight plans were included in the data base. 

The methodology required that data be extracted from the flight plans and 
identified. Appropriate groups of data would then be compared and the 
results presented. Specific procedures peculiar to any one task were 
discussed in the previous sections of this report that discussed the 
results of each task. In general, however, the first and most difficult 
task was to extract the data from the various flight plan formats 
included in the input magnetic tapes. 

Flight plans for Task I were presented in the appropriate output format 
of either the RED or BLUE Airl ines,' while plans for Task III were copied 
on magnetic tape from the teletype output queue and included such 
extraneous data as message switching codes and line control characters. 
These tapes also included flight plans which were not relevant to this 
study. The SRI flight plans in Task II and the AIDS data presented two 
more formats to decode. 
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Computer programs were developed to scan the input tapes and to extract 
and identify relevant data. Data such as date, origin, destination, 
flight time, fuel burn, wind component and temperature were identified 
from each flight plan and stored in files for subsequent comparisons 
between the various flight plan groups. In some cases the data were not 
presented on the flight plans and had to be derived from available data, 
such as winds from the wind correction angle and true air speed on the 
BLUE Airlines flight plans. 

The comparisons between groups of flight plans were then processed by 
computer and the statistical results output in graphic form. Figure 8-1 
is a sample of the typical output format. In each task and sub-task the 
objective was usually to determine differences in some parameter between 
plans based on one set of conditions and plans on some other conditions. 
As Figure 8-1 shows the mean value of the difference was determined, 
along with the variance, standard deviation and 90 percent confidence 
limits. An histogram depicted the frequency of occurrence of incremental 
values of the differences between the two groups. 

8.2 DATA ANOMALIES 

Since the data used in these analyses were collected during 1979 and some 
preliminary processing was conducted by NASA during 1980, it was not 
possible during this project in 1981 to rerun, correct, or recreate any 
erroneous or missing data. As a result of procedural anomalies during 
data collection or other factors which are no longer known, there was 
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Figure 8-1 
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some loss of data for some of the tasks and a corresponding reduction in 
the size of the statistical sample. However, sufficient reliable data 
were left to form valid conclusions from the analyses. 

In Task I, several features inherent in the BLUE Airlines flight planning 
system, the NWS analysis model, and NASA's input procedures caused some 
anomalies in the data base. Three of these were somewhat significant. 

First, an error checking procedure in the NWS software caused pilot 
reports and AIDS data submitted by NASA to be ignored, and resulted 
in the verifying analyses being always identical to the forecasts in 
equatorial regions and the Southern Hemisphere. As a result data from 
some flights in these regions had to be discarded. 

Second, AIDS recreated flights in the BLUE system used direct, great 
circle routes between the origin and the oceanic entry point while the 
minimum time track plans were restricted to airways. This resulted in a 
distance bias generally favoring the AIDS flights which sometimes gave a 
fictitious indication of fuel savings when AIDS flights were compared to 
minimum time tracks. 

Third, since NASA's inputs to the BLUE system tried as closely as 
possible to recreate the original conditions under which AIDS flights 
operated, flight levels were restricted to those used by the AIDS flight 
while minimum time track plans were calculated at optimum levels. This 
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resulted in fuel burn differences which were attributed to the flight 
level difference rather than weather data differences. 

Adjustments were made manually to the computer output to correct the 
findings for these anomalous data. The adjustment factors were deter- 
mined from the actual distance differences between the airways and direct 
routes used by all of the flights in Task I, and from the actual flight 
level differences between the plans. 

Another procedural problem during data collection resulted in a substan- 
tial reduction in the size of the sample in Task III. In Task III AIDS 
observations of winds and temperatures were compared to those on the 
flight plan of the flight which observed those data. This required the 
airlines to retain copies of the flight plans during the data collection 
period. The BLUE Airline accomplished this by periodically dumping the 
output queue from its flight planning system on to magnetic tape. During 
the Task III analysis it was discovered that this "dump" was apparently 
not done frequently enough and substantial numbers of flight plans were 
lost. It was estimated as a result that up to 50 percent of the possible 
segment comparisons were lost because the flight plan corresponding to 
the AIDS- flight could not be found. 

However, sufficient data v/ere still available to form valid conclusions 
and, in fact, a supplemental run which nearly doubled the sample size 
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through relaxed error checking criteria resulted in negligible changes to 
the results. 




pro SPEAS 


